A Bayesian method for calibration and aggregation of expert judgement

Debiasing Expert Overconfidence: A Bayesian Calibration Model

In a decision and risk analysis, experts may provide subjective probability distributions that encode their beliefs about future uncertain events. For continuous variables, experts often provide these judgments in the form of quantiles of the distribution (e.g., 5th, 50th, and 95th percentiles). Psychologists have shown, though, that such subjective distributions tend to be too narrow, represen...

Expert Judgement and Expert Disagreement

As Hammond has argued, traditional explanations for disagreement among experts (incompetence, venality, and ideology) are inadequate. The character and fallibilities of the human judgement process itself lead to persistent disagreements even among competent, honest, and disinterested experts. Social Judgement Theory provides powerful methods for analysing such judgementally based disagreements ...

Incorporating Expert Judgement into Bayesian Network Machine Learning

We review the challenges of Bayesian network learning, especially parameter learning, and specify the problem of learning with sparse data.We explain how it is possible to incorporate both qualitative knowledge and data with a multinomial parameter learning method to achieve more accurate predictions with sparse data. 1 Review of Bayesian Network Learning Constructing a Bayesian network (BN) fr...

Imitation of Expert Judgement

Judgement Swapping and Aggregation

We present the results of an initial experiment that indicates that people are less overconfident and better calibrated when they assign confidence levels to someone else’s interval judgements (evaluator confidences) compared to assigning confidence levels to their own interval judgements (judge confidences). We studied what impact this had on a number of judgement aggregation methods, includin...